On Mon, 04 Jan 1999 22:54:42 +0100, "Thorsten Froehlich"
<fro### [at] charliecnsiitedu> wrote:

>Yes, you are right that the parsing part does not build anything, but please pay
>attention to this little detail:
>Look at the lines you did not mark, what do they do?  They reduce the memory usage!
...
>All these lines of code would have to be written inside the grammar file, wouldn't
>they? And this would make the grammar file a total nightmare!!!  Or how would you
>solve this problem, maybe I am totally wrong!?!

You just call a function that contains the elided bits of code using
the parameters you've parsed out.  It still happens at parse time. 
I'm not too quick on the Bison grammar specification language myself,
but I believe you would just pass the parsed vector into the
"do_scale" function, defined somewhere else, and be on your way.  Yes,
you might have to store the vector somewhere temporarily, but you will
only need to store one such vector for a scale command, and it can
even be in a global variable.

So part of your grammar looks something like this (forgive me if I've
completely botched the Bison syntax; it's been a while since I looked
at it)

OBJECT: OBJECT_TOKEN '{' OBJECT '}' |
        SPHERE |
        CYLINDER |
        #All kinds of other objects
        {$$ = CurObjStackPop();}
        ;
SPHERE: SPHERE_TOKEN '{' VECTOR ',' VECTOR {
          CurObjStackPush( CreateSphere( $3, $5 ) );
        }
        OBJECT_MODS '}'
        ;
OBJECT_MODS: SCALE_STMT |
             ROTATE_STMT
             #whatever else is in object_mods.
             ;
SCALE_STMT: SCALE_TOKEN VECTOR {
              Scale( CurObjStackTop(), $2 ); 
            }
            ;

>Yes, the POV-Ray scene language contains not only data structures but also commands
>(I (still) can't find a better word, it is what you called "do") how to modify this
>data. This is a major difference to the common programming languages which allow easy
>parsing from the top to the bottom, while POV-Ray is self modifying code of some kind
>(or some kind of hidden preprocessor): If you parse C you can parse tokens like a
>stream, nothing (OK, this is oversimplified) you parse later in the stream of tokens
>will require you to *change* data you created earlier.

Nothing in POV changes data created earlier, either, to my knowledge,
except to make fairly localized modifications to the object currently
being parsed.  Some things change global variables; that's simple
enough.  Some things change the parameters of the current object;
that's simple enough too.  Some things look up things that were
defined before in a symbol table and make copies of them into a local
object for further modification.

>* In Bison parser code (or better, in the grammar file) you intended to keep the data
>modifying code out! Now you have to *store* the commands (or actions) that change the
>data. You will then go through all this data later, lookup each object, check if
>there are scale commands for it and execute them. Then you free the commands data
>structures.

No, you just put the data modifying code in function calls, 
implemented in a separate module from the grammar.  Each rule in the 
grammar calls a function in the 'separate module'.  The only
difference is, the code that tells you what happens when a particular
command is found are not polluted with the elements of the grammar,
and the grammar isn't polluted with the code that tells you what it
does when it gets a particular sequence of symbols.

>>You don't extend the Bison generated code; you extend the original
>>grammar definition file. (I will grant you that one would not want to
>>manually generate a parser from a Bison grammar definition file. But
>>then, that's what Bison is for!)
>
>You did not get my whole point: Even a small change to the grammar and the whole
>parser has to be recompiled, I don't want to build a parser from the grammar file,
>just extend the parser code by hand.

Just a small change to the parser and the whole parser has to be 
recompiled now, too.  The only difference is an extra pass in the 
compilation.  Microsoft Visual C++ gives you the ability to specify
additional processors (e.g. a processor that can turn a .y file into a
.c file); I have to assume that any modern IDE allows that.  If not, a
makefile surely does.  If your platform supports neither, perhaps it's

time to get a real OS (no Mac-bashing intended, as I'm sure some 
Macintosh compiler supports at least the IDE method)

>This would require someone in the team to always generate one parser for everybody in
>the team, the only way to make sure the bugs are out everywhere.  If this would not
>be done, each platform developers would also have to make sure that on their platform
>the *whole* parser works as expected - and at least on the Mac we have other things
>to do... :-)

Or, it requires the team to all use the same version of Bison.  Even 
if there isn't a recent port of Bison for the Macintosh, someone could

probably build one fairly easily, especially since you say you can 
work around the command-line limitations.

>(Command line options are another problem: E.g. Macs don't have a command line (but
>of course I can work around this) and getting the same options would still be
>difficult on different platforms.)

Not if someone says "these are the options we will use."  I assume 
everyone on the Team has a way to make a script that enshrines those 
options for all eternity, and maybe even a way to make that script 
part of the official source for their platform.

>>(Incidentally, I believe the C and C++ language standards are just as
>>"open" as the POV-Ray scene language. 
>Also this is an off-topic, I have to inform you that you are wrong:
>Any reason why a standrad document has to be expensive? C++ is a standard, are you
>sure you refer to the ISO 14882 standard document?  I am very convinced it is closed,

You're working from different definitions of open vs. closed.  For the

record, Thorsten meant to say that the C++ standard is fairly fixed, 
not that it was proprietary.  This is in fact true, now.  ANSI C has 
been somewhat fixed for far longer.  On the other hand, neither C nor
C++ is completely reducible to a simple Bison grammar, at least in 
part because of typedefs.  The POV language will have the same 
problem, with #declare and #macro.  Parsing #while will probably be 
tricky, as well.

>Bison does not fit to well into this and I cannot find a switch for the Bison I have
>used which allows C++ code output, but of course this can change in the future! And
>until then the C code is still compatible with C++.

There are other parser generators.  For example, there are BYACC and 
PCCTS, both of which are available in source form.  I'm fairly certain

that if Bison isn't, at least one of them is capable of generating 
parsers in C++.  Some versions of BYACC can even do it in Perl.  Not 
that we currently need either C++ or Perl.  

>>I fail to understand how the work involved in modifying a Bison
>>grammar description file is greater than that involved in modifying
>>PARSE.C.
>
>You need to parse the grammar and only then you can parse the C code, and most modern
>IDEs need plug-in Bisons to to this automatically - there is no old fashioned (but
>very flexible) makefile!

Visual C++ 6.0 has a makefile, but even if it didn't, it's fairly 
simple to add command-line utilities like Bison (possibly surrounded 
by a batch file of some kind) to process files.  Again, since MS is 
rarely state-of-the-art, I assume other vendors have had this for 
years.

>Conclusion: I think Bison is a very interesting, well implemented and useful tool,
>but my position is that it does not fit the POV-Ray needs as well as a handwritten
>parser implementation could (not limited to the current one). Your position is
>different and thats OK for me, so if you find the time to write a POV-Ray grammar and
>implement a sample parser (you don't need to support all objects) and can show me
>that
>I am wrong, then this would be great!

I think that's a worthwhile project, particularly if it's written in
an easily extensible way, can be used as a drop-in replacement for
parse.c/tokenize.c, and is free for use both by the POV-Team and
other, possibly commercial, interests who need a parser.

It's worth noting that there is at least one general-purpose
POV-compatible parser using PCCTS: it's called libparpov, and is part
of the POV-to-RIB converter, which is both quite sophisticated and
quite complete, up to version 3.0.  See
http://www9.informatik.uni-erlangen.de/~cnvogelg/pov2rib/
for more information and for source code.

>PS: Some of my points are Macintosh related simply because I program on the Mac most
>of the time and it is the system I know best.

'sallright.  Some of mine are Windows related for the same reasons.

Post a reply to this message